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Abstract 

We consider stability of shape for a storage model on n nodes. 
These nodes support K. neighborhoods Si C {1, . . . , n} and items arrive 
at the Si as independent Poisson streams with rates Ai, i — 1, . . . , 
K,. Upon arrival at Si an item is stored at node j € Si where j 
is determined by some policy. Let Xj(t) denote the number of items 
stored at j at time t and let X(t) = (Xi(t), . . . , X n (t)) . Under natural 
conditions on the Xi we exhibit simple local policies such that X(t) is 
positive recurrent (stable) in shape. 
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1 The model 



We consider a storage system (or library) with a finite number of nodes 
where identical items are to be stored. The n nodes support non-empty 
neighborhoods Si, i = 1, . . . , JC with 

K 

(J Si = {1, .. . ,n}, 

1=1 

and 1 < JC < 2 n — 1. Items arrive at the neighborhoods as independent 
Poisson processes with rates \ > at Sj, i = 1, . . . , K, where we suppose 
that Y^i=x = 1. Upon arrival at Sj an item is stored at a node j £ Si 
where j is chosen by some policy. We consider local Markov policies where 
each allocation decision is a function of the state, at the arrival time of the 
item, of the neighborhood where the item arrives. We will make this more 
precise below. 

Stability in shape is of interest in several models. There are of course 
various growth models, see for example the crystal growth model studied 
in p], though the methods used there are very different from those we use 
in this paper. Another model which is relevant is a queueing system with 
server vacations or maintenance periods where stability in shape can be 
seen as a fairness criterion for arriving jobs. It is also reasonable to view 
this storage model as a simplified version of the supermarket model (by 
dropping the service) , see for example [6] . 

We have chosen to focus on the routing aspect of this model here. Rather 
more complex phenomena appear when service is considered as well and we 
are investigating a model in which service times are dependent upon both 
the arrival neighborhood and the allocated server. 

We return to the description of our model. Let \Si\ = Ki denote the size 
of neighborhood i and suppose the nodes in Si are enumerated in some way, 
so that Si = {s\, . . . , sj..}. 
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Definition 1.1. We say that j,k G {l,...,n} are neighbors (and write 
j ~ k), if j, k £ Si for some i. 

This equivalence relation can be used to define the graph Q with vertices 
{1, . . . ,n} and edges £, where w = {j, k) G £ iff j ~ k. Our main result 
(Theorem 12. ip needs the following assumption. 

Condition 1.1. The graph Q is connected. 

If the Condition 11.11 is not fulfilled, then we have two or more discon- 
nected components, that is, sets of nodes such that arrivals to one of these 
sets cannot be routed to the other. In this case, it is impossible to obtain 
positive recurrence in shape, for any routing policy. If the number of dis- 
connected components is at least 4, then even null-recurrence is impossible. 
The reasoning for this is similar to that in the argument immediately after 
Theorem [2TJ 

Denote the configuration of the system at moment t by 

X(t) = (X l (t),...,X n (t)), 

where Xi(t) is the number of items stored at node i at time t. The center 
of mass or average load of the configuration is 

n 
i=l 

and we denote the shape of the configuration by 

X(t) = (X 1 (t),...,X n (t)) = (X 1 (t)-M(t),...,X n (t)-M(t)), 

the vector of loads relative to the center of mass. Note that, if a new item 
arrives at time t, then M(t) = M(t—) + -. Also, if we know the shape X(t), 
it implies that we know which node is minimally loaded and we know the 
load differences between the nodes (as Xi(t) — Xj{t) = Xi{t) — Xj(t)). 

Obviously, the process X(t) is Markovian for any decision rule that de- 
pends only on the current node loads. In order for the process X(t) to be 
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Markovian, we require that the decision of choosing the node is made accord- 
ingly to some decision rule which depends only on the current shape of the 
system. Also, we are mainly interested in local decision rules, that is, if an 
item arrives to the set Si, then the only information about the configuration 
of the system that can be used to make a decision is what happens in the set 
Si. For example, the decision can be based on the differences Xi(t) — Xj(t), 
1,3 G Si. 

If the decision rule is configuration independent and time homogeneous 
this gives rise to a space homogeneous (n — l)-dimensional random walk, 
which is transient for n > 3 and at best null recurrent for n < 3. Therefore, 
if one wants positive recurrence in shape, the decision rule must depend on 
current configuration. Of course, all nodes must receive arrivals for ergodic- 
ity in shape to be achieved, hence the walk cannot live in a lower dimensional 
sub-space. So, our goal is to find a rule for redistributing the arriving items 
at each moment of time in a way to have positive recurrence in shape. One 
of the possible choices is to send the item to the node with minimal load Si 
(Join the Shortest Queue routing policy). 

We present four routing policies. Two ensure the same rate of the arrivals 
to different nodes, and the two others guarantee stability in shape, if some 
explicit conditions are fulfilled. We note also that the conditions we refer to 
can be easily checked in practice and the implementation of routing policies 
we propose is algorithmically simple. 

The paper is organized as follows. In Section [2] we introduce the no- 
tations, define the routing policies and state the results. In Section [3.11 we 
formulate the known facts we will use in our proofs. In Section \3. 21 we prove 
Theorem 12.11 for which we need some auxiliary lemmas, and then we prove 
Theorem 12.21 In Section 13.31 we first prove a lemma that translates the 
condition of Theorem 12.31 into the language of convex analysis, and prove 
Theorem 12.31 
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2 Results 



Let us first introduce some notation. For i = 1, . . . , K, denote by Aj the set 
of points pW = (pW , . . . , p$ ) G R Ki such that 

p^ > for j = 1,. . . ,m, 

m 2l1 ) 

We say that p = (pW , . . . G M KlH hK ' c is non-negative if 

p^ > for j = 1, . . . , Ki and i = 1, . . . , /C, 

and positive if 

p^ > for j = 1, ... , Kj and i = 1, . . . , /C, 

By i 7 denote the linear transformation that takes a point x 6 M™ to the 
point y G M™ such that y% = Xi — \ Y^Li for i = 1, . . . , n. In words, the 

n — *j — i j 

point y represents deviations from the center of mass for the configuration 
x. Let JOT = {y G R n : J2i=i Vi = 0}- Note that F(R n ) = JOT. 

Let = F(X(t)). Since F is a linear transformation and JOT is a 

(n— l)-dimensional subspace of R n , we can say informally that the dimension 
of the process X(t) is 1 less than the dimension of X(t). Since the state 
space of the process X(t) is N ra C W 1 , we see that the state space of the 
process X(t) is F(W l ) C JOT. 

A point x = (x\, . . . ,x n ) G N n represents the load of the system. By 
x {Si) denote the load of the nodes in Si. Let 1 be the vector with all ones: 
1 = (1, . . . , 1) G N n . 

Now we define the notion of routing policy (RP). 

Definition 2.1. A routing policy P is a function that takes a configuration 
x G W to a point P(x) = (p (1) , . . . ,p (,C) ) G suc h that G 

Aj,i = 1, ...,/C. For the process X(t) (or X(t)) with routing policy P, 
an item arriving at neighbourhood Si, when the configuration of the system 



5 



is x, is routed to node s* with probability p - (x) . The decisions are made 
independently for each arrival. 

For the process X(t) to be Markovian, we suppose that all routing poli- 
cies satisfy the following. 

Condition 2.1. The routing policy P depends only on the current config- 
uration shape, that is, for any x and y such that F(x) = F(y), we have 
P(x) = P(y). 

There is an equivalent way to write down this condition formally: for 
any c € Z and any x and y such that x = cl + y, we have P(x) = P{y). 

The decision about routing can be made using the complete information 
about configuration shape, or only partial information: 

Definition 2.2. Fori = 1, ... ,/C, denote by the composition of routing 
policy P and projection on M Ki , i.e. for any x, we have pW(x) = p^ when- 
ever P(x) = (p^\ . . . ,p^). We say that a routing policy P is local if, for 
i = 1, ... ,/C, the function depends only on the load of the nodes in Si: 
for any x and y such that = y^ Si \ we have pW(i) = pW(y). 

Recall that we demand that a routing policy depends only on current 
configuration shape. In the case of local routing policy this means that, for 
i = 1, ... ,/C, the function pW depends only on current configuration shape 
restricted to Sf. for any c G Z and any x and y such that x^^ = cl( Si ^ +y( Si \ 
we have pW( x ) = P«(y). 

In this paper we will consider four local routing policies. 

Definition 2.3. An equilibrium routing policy (ERP) is a routing policy P 
such that P does not depend on x and the resulting arrivals at all nodes are 
independent Poisson processes with the same rate 

n n 

(recall that Ylf=i = V- 
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Definition 2.4. A strong equilibrium routing policy (SERP) is a routing 
policy P such that P does not depend on x, the resulting arrivals at all the 
nodes are independent Poisson processes with the same rate 

Ef = i A, _ 1 
n n 

and P is positive for all x. 

Let us consider the following system of linear equations: 



YTjLi a ij =\ for i = 1, . . . , /C, 



(2.2) 



, n, 



where 



St; 



1 ii £ = m, 
if £ / m. 

Remark 2.1. The system (|2.2p is a special case of the maximum bipartite 
matching problem and necessary and sufficient conditions for existence of 
positive/non-negative solutions are well-known. 

For each non-empty collection of neighbourhoods J C {1,2, ...,/C} let 
Sj = Uj^jSj and let nj denote the number of nodes in Sj. Then, 

^2\j<nj/n for all J C {l,...,n} (2.3) 



is necessary and sufficient for existence of non-negative solutions to (|2.2p . 
Strict inequality in (|2.3p for all J except and {1, 2, . . . TC\ is necessary and 
sufficient for the existence of positive solutions to (|2.2p . 

Indeed, if (|2.3p is not satisfied, then at least one node in some Sj must 
receive items at rate greater than 1/n, under any routing policy. The suf- 
ficiency can be shown using maximum-flow minimum-cut method (cf., for 
example, Q 

Remark 2.2. Note that for any parameters of the model Si,... ,Sjc and 
Ai, . . . , A n we have: 
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• there exists an ERP iff ()2.2p has a non-negative solution; 

• there exists a SERP iff (|2.2p has a positive solution. 

Indeed, if (|2.2p has a non-negative /positive solution we can define p^ = 
otij/\i. If we have an ERP /SERP, then aij = \pf is a non-negative/positive 
solution of (|2.2p . 

We also rewrite this statement in the language of convex analysis (see 
Lemma 13, 5p . 

As solving (|2.2p is a problem of linear programming, the existence of 
SERP can be easily checked in practice. 

Example 2.1. • Consider a system with n = 3 nodes and all possible 
neighborhoods of size 2, X\ + A2 + A3 = 1. Then, there exists a positive 
solution of the system (|2.2p iff Aj < 2/3 for i = 1, 2, 3. 

• Similarly, for n = 4 and all possible neighbourhoods of size 1, there 
exists a positive solution of the system (|2.2p iff Xj < 1/2, j = 1, . . . , 6, 
and X^jej -\? < 3/4 for all J such that nj = 3. 

Now we define the other two routing policies which we study. For x G N n , 

let 

s 1 a (x) = max < s 1 a G & : x ol = max {x„i} > (2.4) 

and 

s'- . (x) = min<^ s* G : x„» = min {x„i} [. (2.5) 

Jmln I 3 S J l=l,...,Ki S l ) 

In words, for any load of the system x G N n , s j min ( 2; ) is the first node in Si 
such that in this node the load is minimal, si (x) is the last node in Sj 
such that in this node the load is maximal. 

Definition 2.5. Join the Shortest Queue (JSQ) routing policy is the routing 
policy P(x) = (p^\ ■ ■ ■ ,p^), where 



P., 



otherwise. 



S 



Definition 2.6. Suppose that there exists a positive solution aij of (|2.2p . 
Let < e < mm a^. We define e-perturbed strong equilibrium routing 
policy (e-PSERP) as P{x) = (pW, . . . ,p ( - K) ), where 



If Ki = 1 (i.e., the neighborhood Si has size 1), then we have no freedom to 
choose probabilities and Pj(x) = 1 for any x. 

Note that in each of the four cases the routing policy can be chosen to be 
local. Indeed, in the case of JSQ it is clear immediately from the definition. 
In each of the other three cases, we first need to note that we can choose 
the same solution of (|2.2p for all x £ N n , then it is easy to see that the 
corresponding policy is local. Moreover, in the cases of ERP and SERP it 
does not depend on x. 

We study the behavior of the process X(t) = (Xi(t), . . . ,X n (t)), where 
Xi(t) = Xi{t) - M(t), M{t) = i Er=i x i(0> which represents the shape of 
the system. In order to simplify the notation, we prefer to keep the same 
symbol for the process with any RP; instead when dealing with X(t) or X(t) 
we will state explicitly which RP is used. 

Let {X e (m)} m< zn (resp. {X e (m)} m ^) be the embedded Markov chain 
for the process {X(t)} t >o (resp. {X(t)} t >o), obtained when we look at 
the system only at the moments of arrivals. Note that {X e (m)} mg N and 
are indeed Markov chains, as the arrivals are Poisson. Note 
also that {X e (m)} me ^ has period n under any of the policies considered 
(indeed, if X e (m) = x, we need the same number of items to arrive at every 
node to obtain X e (m') = x, so we must have w! = nl for some I). For 
EPR, SERP and e-PSERP the process {X e (m)} mg N is irreducible, as all 
nodes have positive arrival rates and thus any shape can be obtained from 
any other shape. The situation is more delicate for JSQ routing policy. 




p)'[x) = < 



if s) = s) . (x), 
if s) = s\ (x), 



otherwise. 
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For example, with JSQ, if node j does not belong to a neighborhood of 
size 1, then starting from configuration X e (0) = it is impossible to obtain 
configuration with Xf(m) = x for all i ^ j and X|(m) = x + 2. It important 
to note, however, that the configuration X e (m) = is reachable from any 
configuration. 

By r denote the time of the first return to the origin: 

r = inf{m > : X e {m) = 0}. (2.6) 

We say that 

(a) {X e (m)} m€N is transient if P(r = oo | X e (0) = 0) > 0, 

(b) {X e (m)} mG N is recurrent if P(r < oo \ X e (0) = x) = 1 for any x, 

(c) {X e (m)} m< zfq is positive recurrent if E(r | X e (0) = x) < oo for any x. 

This is of course a slight abuse of notation, since, strictly speaking, only the 
process restricted to the irreducible class of can be (positive) recurrent. 
Nevertheless, we prefer to give the definition in this form because, as we will 
see below, (b) and (c) either hold for all or for no x (so that the results are 
more complete). 

Since the rates of our processes are bounded away from and oo, positive 
recurrence of {X(t)} t >o is equivalent to positive recurrence of {X e (m)} me pj. 
So, we will prove the results for {X e (m)} m ^. 

Theorem 2.1. Suppose that Condition M . l\ is satisfied and there exists a pos- 
itive solution of (|2.2p . Suppose that we construct the process {X e (m)} me N 
using either JSQ routing policy or e-PSERP. Then X e {m) is positive re- 
current. Moreover, there exists c > such that for all < d < c we have 
E(e c ' r | X e (0) = x) < oo for all x. 

Note that using ERP or SERP it is impossible to have positive recur- 
rence of X e (m). Indeed, these routing policies provide independent Poisson 
arrivals with the same rate to all nodes. Then the behavior of the shape can 
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be described by a (n — l)-dimensional random walk with zero drift, which 
is transient if n > 3 and null-recurrent if n < 3. 
Define the shape magnitude as 

n n 

nix®) = y, Ut? = - M (*)) 2 ( 2 - 7 ) 

i=l i=l 

(so Q{X(t)) is in fact the square of the Euclidean norm of X(t)). 

Remark 2.3. From the proof of Theorem \2.1\ (equations {3.$ and LS.8\ )) 

it can be extracted that JSQ routing policy minimizes the expected shape 
magnitude, that is, for any routing policy we have 

W ny ^^(^(m + 1)) | X* (TO,) = x) 

> E J6 ' Q (S(X e (m + 1)) | X e (m) = x). 

We also have the following converse results (in some sense) to Theo- 
rem 12.11 Note that in Theorems 12.21 and 12.31 we do not require the routing 
policy P to be local. 

Theorem 2.2. Fix the parameters of the model: Si,...,S/c, \i,...,\jc- 
Suppose that there exists a routing policy P such that the process X(t) with 
the routing policy P is recurrent. Then there exists a non-negative solution 
of (|2,2p (and thus for the model with these parameters there exists an ERP). 

We can also rewrite Theorem 12.21 in a different way: 

Corollary 2.1. Fix the parameters of the model: S±, . . . , Sic, Xi, . . . , \/c- 
Suppose that there is no non-negative solution aij of the system (|2.2p . Then 
for any routing policy P, the process X(t) with the routing policy P is tran- 
sient. 

Theorem 2.3. Fix the parameters of the model: Sx, ■ ■ ■ ,Sjc, Ai,...,A/c. 
Suppose that there is no positive solution a™ of the system (|2.2p . Then 
for any routing policy P, the process X{t) with the routing policy P is not 
positive recurrent. 
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The following problem is still open. Fix the parameters of the model: 
Si, . . . , Sjc, Ax, ...,\jc- Suppose that there is no positive solution ay of 
the system (|2.2p . but there exists a non-negative solution. Under which 
conditions on the parameters of the model Si, ... , Sic, X\, . . . , X/c (and n) 
does there exist a (local) routing policy P such that the process X(t) with 
the routing policy P is recurrent? 

3 Proofs 

The structure of this section is as follows. First (Section 13. ip we formu- 
late some known fact which we will use in our proofs. In Section 13.21 we 
introduce some notations and define two functions (/ and g) we will use 
to prove Theorem 12.11 Then we prove four lemmas, obtaining bounds on 
E(/(X e (m + 1)) - f(X e (m)) | X e (m) = x) for JSQ and e-PSERP. Using 
these bounds, we prove Theorem 12.11 Then we prove Theorem 12.21 In Sec- 
tion 13.31 we first recall some definitions from complex analysis and apply 
these to our model. Then we prove Lemma 13.51 which translates the condi- 
tion of Theorem 12.31 into the language of convex analysis, and then we finish 
the proof of Theorem 12.31 

3.1 Preliminaries 

We state some known results that we will use in our proofs. Note that The- 
orems [XT] and E21 are Theorems 2.2.3 and 2.2.6 respectively from [3], where 
we use 'positive recurrent' instead of 'ergodic'. This change is necessary 
as our Markov chains are periodic. That the results also hold for periodic 
chains is mentioned in Section 1.1 of [3j. In fact, to see that the reformulated 
theorems are valid it suffices to consider the Markov chain % at embedded 
instants I = k + pr, where p is the period of the chain and A; is a fixed 
number. 

Let us consider a time homogeneous irreducible Markov chain r/ m with 
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countable state space TL. 

Theorem 3.1. The Markov chain rj m is positive recurrent if and only if 
there exists a positive function f(x),x G TL, a number e > and a finite set 
A S TL such that for every m we have 

E(/(r?m+l) - f(Vm) \Vm = x) < -£, X $ A, (3.1) 

E(/(r? m +l) \Vm = x) < do, ieA 

Theorem 3.2. For i/ie Markov chain r] m to be not positive recurrent, it is 
sufficient that there exists a function f(x),x £ TL, and constants C and d 
such that 

• for every m we have 

E(f(rj m+ i) - f(rj m ) | r] m = x) > 0, x £ {/(x) > C}, 
where the sets {x \ f{x) > C} and {x \ f(x) < C} are non empty; 

• for every m we have 

E (l/0?m+l) - f(Vm)\ I Vm = X) < d, X £ TL. 

The following theorem is an immediate consequence of Theorem 2.1.7 
from [3]. 

Theorem 3.3. Let (0,,^, P) be the probability space and {J- n , n > 0} be 
an increasing family of a -algebras. Let {6;, / > 0} be a sequence of random 
variables such that &i is T {-measurable, and So is a constant. Let 

Uk+i = ®fc+i - 6^. 
If there exist positive numbers e, M , such that for each k we have 

E[yfc+i | Fk] < -e, 
|y fc+ i| < M a.s., 

then, for any 5\ < e, there exist constants C = C(©o) and ^2 > 0, such 
that, for any m > 0, 

P[6 m > -Jim] < Ce~ S2m . 



13 



3.2 Proofs of Theorems O and [2^21 

To prove Theorem 12. 1\ we need some additional notations and four lemmas. 

Suppose that we are using either JSQ routing policy or e-PSERP to 
construct the process X e (m) (for now, it does not matter which one). We 
are going to construct a supermartingale with bounded jumps, that will 
allow us to obtain exponential bounds on r (see (|2.6j) for the definition of 
r) and thus to prove positive recurrence of X e (m). 

Let 

f(X e (m)) = f(Xf(m), . . . , X e n (m)) = £(X?(m) - M(m)f = £(l e (m)), 

i=i 

where Q(X e (m) is the shape magnitude defined in (|2.7p and 

1/2 



/(X e (m)) = V/(^ e M) 



,i=l 



We will prove that g(X e (m)) is a supermartingale with bounded jumps. To 
do that, we will need some auxiliary lemmas. In Lemmas 13.11 and 13.21 we 
estimate ~E[f(X e (m + 1)) — f(X e (m)) \ X e (m) = x] in terms of X e (m) 
for e-PSERP and JSQ respectively. In Lemma 13.31 we obtain a bound on 
\f(X e (m + 1)) — f(X e (m))\, which is needed for the proof that g(X e (m)) 
has bounded jumps. 

First, we introduce the process (Yi(m), . . . , Y n {m)) obtained when the 
item that arrives at Si is directed to node s* with probability = aij/Xi, 
j = 1, . . . Ki (that is, using SERP). The processes X e (m) and Y(m) are 
defined in the same probability space, use the same arrivals, and if X e (m) = 
Y{m) = x, then X e {m + 1) and Y(m + 1) are obtained from x using the 
respective routing policies. 

Using the fact that a^s are such that arriving items are routed to node 
i with probability 1/n for any i, we have 

E[(Yi(m + 1) - M(m + l)) 2 - (Y^m) - M(m)f \ Y{m)} 
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1 (jYi(m) + 1 - M(m) - ^ - (Y^m) - M(m)) 2 ) 
+ ((i;-(m)-M(m)--J - (YAm) - M(m)) 



n \ \ n 
1 1 

~ ~ "72 



(3.2) 



E[/(y(m + 1)) - f(Y(m)) \ Y{m)} = n (--\)=l--. (3.3) 



n 

as M(m + 1) = M(m) + |. Thus, 

1 1 \ 1 

2=1-- 

Denote by Cj the event that an item arrives at set Sj. Recall (|2.4p 
and (|2.5p . From now on, instead of writing s* max (JT e (m)) and s* m . n (X e (m)), 
we will write iust s*- and s*- 

Jmax Jmin 

Lemma 3.1. Suppose that the process X e (m) is constructed using e-PSERP. 
Then 

E[f(X e (m + 1)) - f(X e (m)) \ X e (m) = x] 

= -2 £ V(XJ (m)-X e sl ( m )) + l-I. (3.4) 

Jmax ^min n 

Proof of Lemma \ 3. 11 Suppose |<Sj| > 1. We have 

E[f(X e (m + 1)) - f{Y{m + 1)) | X e (m) = Y(m) = x, Q] 

= E[yi e ,(ra + 1) 2 -F sl (ra+1) 2 | X e (m) =Y(m) = x, Q 
J=l 

= fffc ( m ) + l-M(m)--) 2 

+ E (^H-M(m)-i) 2 ) 

3 ft till II 

(m) + l-M(m)-i) 2 

Aj \ V Jmax 71/ 

+ E (^h-mh4) 2 ) 

3 ft mm 

= -£(J$ (m)-X* (m)) (3.5) 

Aj Jmax Jmin 
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as we conditioned on X e (m) = Y(m) = x. Thus, 



E[f(X e (m + 1)) - f(X e (m)) | X e (m) = x, Q] 

= E[f(X e (m + 1)) - f(Y(m + 1)) | X e {m) = Y{m) = x, d] 

+E[/(F(m + 1)) - f{X e (m)) \ X e (m) = Y{m) = x, C t ] (3.6) 
= -^(JtJ (m)-JtJ. (m)) 

+E[/(y(m + 1)) - /(T(m)) | Y(m) = x, Q] 

and 

E[/(X e (m + 1)) - f(X e (m)) \ X e {m) = x] 
K 

= £ mf(X e (m + 1)) - f(X e (m)) | X e (m) = x, Q] 
i=l 

= -2 £ V(r, (m)-A e 8 ( m )) + l-_. (3.7) 

jmax imin Tl 

1=1 

Note that if there is a neighborhood of size 1, by Condition II. II it should be 
subset of another neighborhood, of size at least 2. As the terms correspond- 
ing to neighborhoods of size 1 in (|3.7|) will be equal to 0, the equation (|3.7|) 
still holds. Lemma [3.11 is proved. ■ 

Lemma 3.2. Suppose that the process X e (m) is constructed using JSQ rout- 
ing policy. Then 

E[/(A e (m + 1)) - f(X e (m)) | X e {m) = x] 

" 2 E E ^.H"^ (m)) + l--. (3.8) 

Proo/ of Lemma \3.2[ Analogously to (|3.5h . 



E[/(A e (m + 1)) - f(Y(m + 1)) | X e (m) = Y(m) = x, Q] 

= E H + i-MH-^ 2 

. , . A;, \ \ Jmin It ' 

3 / 3 min 
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So, 



m 1 n 

1\ 2 / 1\ 2 
Y s i(m) + 1 - M(m) J + 2^ lY^m) - M(m) 

E T^fc M + l-M(m)-i) 2 +te(m)-M(m)-i) 2 
.77" Aj vv Jmin n/ \ o n/ 

1\ 2 / 1\ 2 

y si (m) + 1 - M(m) + ( Y.i (m) - M(m) 

j nJ \ Jmin n 

• £ ^(^(m)-^ (m)). (3.9) 

JT^Jmin 



B[f(X e (m + 1)) - /(X e (m)) | X e (m) = x, C t ] 

= E ^TOM-J^. M) 

+E[/(y(m + 1)) - f{Y{m)) \ Y{m) = x, Q] (3.10) 



and 



E[/(X e (m + 1)) - /(X e (m)) | X e (m) = x] 
K 

= W(X e (m + 1)) - /(X e (m)) | X e {m) = x, Q] 

" 2 E E (m)) + l--. (3.11) 

. 1 . t . 3 ^min <l 

Lemma 13.21 is proved. ■ 

Denote by the i-th coordinate vector, i = 1, . . . ,n. The next lemma 
will be used to bound jumps in / due to any possible one-step changes to x. 

Lemma 3.3. Let x £ N n andm(x) = \ YTj=i x j- 7 /EI=i (xi-m(x)) 2 > 0, 
then for each ei, i = 1, . . . , n, 

|/(x + ei)-/(x)|<4v7(i). (3.12) 
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Proof of Lemma \3.3[ Without loss of generality, consider the first coordinate 
vector e\. We have then 



f(x + ei) 



1\ 2 v-^ / 1\ 2 

x\ + 1 — m(x) J + 2^ ( Xi — m(x) J 



(xi — m(x)) 2 + 2(xi — m(x)) ^1 ^ + ^1 ^ 

+ (xi - m(x)) 2 - - ^ (xi - m(x)) + 
i=2 t=2 
n 1 

(xj — m(x)) + 2(xi — m(x)) +1 



i=l 



as 



1 - 

- Jj (a* - m(x)) = 0. 



i=l 



Hence for each i = 1, . . . , n we have 



f(x + ei) - f(x) = 2(xj - m(x)) + 1 



It remains to show that, if Ya=1 ( x * ~~ m(x)) 2 > 0, then 



(3.13) 



(3.14) 



2(xi - m(x)) + 1 - - < 4 x //(xj. 

Note that Y^n=i ( x * ~~ w-(x)) 2 > implies that there exists i,j such that 
\%i — Xj\ > 1. Thus, there exists at least one I such that |x; — m(x)| > 1/2 
which implies that y /(x) > 1/2. So, 

|/(x + e 4 )-/(x)| <2|xi-m(x)| + 1 



n 

<2[5^(zi-m(x)) 1 



1/2 



+ 1 



i=l 



<4v7R- 



Lemma 13.31 is proved. 
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Lemma [3.31 implies that, for any RP, if Ya=i (^f( m ) ~ M(m)) 2 > 0, 



then 



\f(X e (m + 1)) - f(X e (m))\ < 4V/(^ e (m)) = Ag{X e {m)). (3.15) 

It is important to note that the next computations are valid for JSQ and 
for e-PSERP. 

Lemma 3.4. There exist c<i > and a > 0, such that for all x S N n with 

1 " I 

Xj : / X'j | a 



max 



n " ■ 



E(/(X e (m + 1)) - f(X e (m)) \ X e (m) = x) < -c 2 ^f(xj. (3.16) 
Proof of Lemma \3.4\ We have 

n 

f(X e (m)) = ^(Xf(m)-M(m)) 2 
1=1 
K 

i=l je5i 
/C 

< E max{(X?(m) - M(m)) 2 } 

< n E m ax(X|(m) - M(m)) 2 . 

1=1 

We now show that under Condition 11.11 we have 

Emax(X|(m)-M(m)) 2 < C3 (E(^ (m))) (3.17) 

1=1 1=1 

and also 

Emax(Xj(m)-M(m)) 2 <c 4 (V £ n, ; !.V; .Y; (m))) . 

2=1 *=1 3T3w 

(3.18) 
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Let us consider (|3.17p . If X% (m) < M(m) < X% (ra), then, obviously, 

Jmin J max 

(X?(m) - M{m)) 2 < {X e gl (m) - X e gi (m)) 2 . Suppose that M(m) < 

jmax j m i n 

(m) (the case M{m) > X e t (m) can be treated analogously). Con- 

Jxnin jmax 

sider the sets of nodes {j : X?(m) < M(m)} and {j : Xj(m) > M(m)}. By 
Condition 1.1 some neighbourhood contains nodes from each of these sets 
and hence there exists i* such that X% (m) < M(m) < X e it (m) and 

Jmin imax 

a sequence of neighbourhoods indexed by io = i,ix, ■ ■ ■ , ik = i* such that 
5 i; _ i nS ll / 0, I = 1,... ,k. Thus, 



max|(Xj(m) - M(m)| < X% (m) - M(m) (3.19) 

Jmax 

< X 6 , (m)-X e i (m) + X£ (m)-M(m) 

Jmax Jmin Jmin 

< X e 8 (m)-X£ (m) + X\ (m)-M(m). 

Jmax Jmin jmax 

The last inequality is due to the fact that, as Si n ^ 0, it holds 

x % (m) = min Xf* (m) < min X% (m) 

< max X e t (m) < max X e i (m) = X e ; (m) 

sjeSfl-lSi! S J S \S„ S J S imax 

J J 1 

Continuing (|3.19p . we get 



jes. : 1 * 3 



max | (X, e (m) - M(m) | < X£ (m) - M(m) 

Jmax 

< X* {m)-X e sl {m)+X\ (m)-M(m) 

Jmax Jmin ^Jmax 

< X e s% {m)-X e s% (m)+X\ {m)-X e sH (m) 

Jmax -?min Jmax Jmin 

+X% (m) - M(m) 



Jmax 



and so on until ik = i* (at the last step one has to use X% (m) < M(m)). 

Jmin 

So, we obtain 



max | (X](m) - M(m)\ < ( m ) ~ M) 

• ?G5 '* ;_q S Jmax S j'min 

and (|3.17|) follows with some C3 < k. The argument for (|3.18p is similar. 
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Then Lemma O together with (|3TT7j) (for e-PSERP), and Lemma [372] 
together with (|3.18p (for JSQ), imply that, for some c 2 > 0, 



B(f(X e (m + 1)) - f(X e (m) \ X e (m)) < -c 2 V/(* c M) = -c 2 g(X e (m)), 

(3.20) 

when f(X e (m)) is large enough. Lemma 13.41 is proved. ■ 

Proof of Theorem \2.1\ First, we verify that g{X e {m)) has bounded 
jumps. If Y£=i i X i( m ) ~ M(m)) 2 = 0, then, obviously, g(X e (m + 1)) - 
g(X e (m)) < const. So, suppose that £^=i {Xf{m) - M(m)f > 0. 

Using inequality + b — 1| < |6| for b > —1, we obtain that 



\g(X e (m + l))-g(X e (m))\ 



= [fiX^m))] 1 ' 2 
< [f(X*(m))]V 2 



f(X%m + l))- f(X e (m)) 



1+ f(X e (m)) 
f(X e (m + l))-f(X e (m)) 



1/2 



f{X*{m)) 
\f(X e (m + l))-f(X e (m))\ 



[/(^(m))]V2 



(3.21) 



< 4, 

by Lemma 13. 
Let 



A = F(N n ) n {x £ R n : max \ Xi \ < a}, 



where a is from Lemma l3.4i That is, A is the set of possible configurations 
of X e such that 
now prove that 



of X e such that maxj = i ... n \Xf\ < a. Note that the set A is finite. Let us 



B[g(X e (m + 1)) - g(X e (m)) \ X e (m) = x] < -c 2 /V2, 



if x i A. Indeed, if x G F(N n ) \ A, as y/TTb < 1 + | for b > -1, we get 



(using Lemma [37 
E[ 5 (X e (m + 1)) - g(X e (m)) \ X e (m) = x] 



21 



E 



1 + 



f(X%m + l))-f(X e (m)) 



< 



< 



f(X*(m)) 
B[f(X e (m + 1)) - f(x) | X e {m) = x] 



1/2 



2^ 



1) | X e {m) = x 
(3.22) 



£2 
2 ' 



Thus, by Theorem 13.11 the process X e is positive recurrent. 
For ta = inf{m > : X e {m + k) £ A}, take now 



6, 



g(JT e (m)), if m < t a , 
— (m — ta), if m > 



and apply Theorem 13.31 to the sequence {@ m }. We have that for any 5i < 
C2/2, there exist C and 82 such that 

P[t4 > (1 - SJm I X e (/fc) = x' $ A] < Ce~ S2m . 

Note that there exist k and 5 > such that for any y G A 

P[X e (m + = for some I < k | X e (m) = y] > & 

It is then not difficult to obtain that E(e c ' T | X e {k) = x) < 00, where 
r = inf{m > : X e (m) = 0}. Theorem 12.11 is proved. ■ 

Proof of Theorem \2.2[ Let Ni(t) be the number of arrivals at Si by time t. 
Since Ni(t) is a Poisson process with rate Aj, a.s. iVj(t) — > 00 and Ni(t)/t — > 
Aj as t — >• 00, i = 1, . . . , n. 

As X(i) is recurrent, we have that for almost every realization of the 
process X(t) there exists an infinite sequence t\,t2, ■ ■ ■ such that X(tj) = 
for all j. For these moments tj we can define 

hN ik (tj) 



Oiik{tj 



Ni(tj) 



where Ni^(tj) is the number of items arrived at Si and routed to node s\ 



by time tj. So, sending the proportion 



<*ik(tj) 

A, 



of items arriving at Si to 
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si, results in the same number of items at all nodes. As the sequence of 
otik(tj) is bounded, we can chose a subsequence aik(tj') - ► ®ik, as tji — ► oo. 
Evidently, > and EfcLi a ik = \- Then, as 



-i n 1 ^ 



n i — ' n 

l'=l i=l 



we obtain 



1 Xdtj 
n 



El 






i 


Ef= 






i 


Ef= 








Ef= 





/c 



i=l m=l MjJ 

1=1 J m=l 



As 



— — ► A,- and 



and ai m (tji) — > Oj m , we see that {aj m } is indeed a solution of (12. 2|) and 
Theorem 12.21 is proved. ■ 

3.3 Proof of Theorem EH 

We will use theorems from [TJ, therefore let us recall some definitions from 
there. A subset C in R n is called convex if (1 — X)x + Ay G C for every 
x £ C, y £ C and < A < 1. A subset M in R n is called an affine set if 
(1 — X)x + Xy G M for every x G M, y G M and A G M. Given any set 
A C R n there exists a unique smallest affine set containing A (namely, the 
intersection of the collection of the affine sets M such that A C M) , this set 
is called affine hull of A and is denoted by aff A. Given a set A C M n the 
interior that results when A is regarded as a subset of its affine hull aff A 
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is called relative interior of A and is denoted by ri A The closure of A is 
denoted by clA Note that cl(clA) = clA and ri(riA) = nA; moreover, if 
A is convex, then cl(rivl) = cl^4 (see Theorem 6.3 in [7]). If A is convex and 
A ^ 0, then riA ^ (see Theorem 6.2 in [7]). A set A is called relatively 
open if ri A = A. 

Let us apply the definitions to our model. Note that Aj £ is convex, 
i = 1, ...,/C. By 12 denote the linear transformation that takes a point 
p = (pC 1 ),...,^) G R K i+-+ K 'c to the point x = (x ly . . . , x n ) £ R n such 
that 

xe = ^2^2XiPj6 es i for £=l,...,n 

i=l j=l 

where, as before, 

1 if I = m, 

Sim 

'0 if^^m. 



Let L := E(A 1 x • • • x A/c) C R n and (3.23) 
D := F(L) cMcR n . 

Since, for i = 1, . . . , /C, the set Aj is convex, we see that the set Ai X • • • X A^ 
is convex (see Theorem 3.5 in jJJ). As E and F are linear transformations, 
the sets L and D are convex (see Theorem 3.4 in [7J). Since, for i = 1, . . . , IC, 
the set Aj is compact, the set Ai x ■ ■ ■ x A^; is also compact. Since E and 
F are linear transformations, and therefore, continuous transformations, we 
see that the sets L and D are compact. In particular, D is closed, that is, 
clD = D. 

To translate the condition of Theorem 12.31 to the language of convex 
analysis, we need the following lemma. 

Lemma 3.5. For any parameters of the model Si, . . . , Sk, and Ai, . . . , A n , 
the following statements are equivalent: 

1. € riD; 
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2. there exists a positive solution ctij of the system (|2.2p . 

Proof. Note that ri A, is the set of points = (p® , • • ■ , Pk} ) £ such 
that 

pf>Q for j = l,..., Ki , 

Moreover, 

ri(Ai x • • • x Ajc) = (riAi) x • • • x (ri A*) (3.25) 

(see the proof of Corollary 6.6.1 in [7]). Since FoE is a linear transformation, 
we see that 

F(£(ri(Ai x • • • x A*;))) = iiF(E(A l x • • • x A*;)) = riD 

(for the first equality see Theorem 6.6. in [7]). 

Thus we have proved that F(E((ri Ai) x • • • x (riAx;))) = riD. Therefore, 
y = F(E(p)) £ riD if and only if p e (ri Ai) x • • • x (riAx;). Recalling $£2M 
for ri Ai, we get that y = F(E(p)) £ riD if and only if 

p^P > for j = 1, . . . , k» and i = 1, . . . ,K, 

3 n (3.26) 
Yg =1 pf = \ for i = 1,...,/C. 

Suppose that item 1 holds, that is, G riD. Then there exists p £ 
and x g flj n suc h that p satisfies (pT26l) . £?(p) = x and F(x) = 0. 
Then we have 

n n K. Ki K, Hi K. 

£**=£££ **?\.s = £ £ = £ ^ = i. 

£=1 £=1 i=l j=l i=l j=l i=l 

In the first equation we use the definition of E, in the second the fact that 
J21=i si = 1j ™ * ne third the second line from (|3.26[) , Therefore, using the 
definition of F, it follows from F{x) = that x = E(p) = (— , . . . , ^). Then 
Syli ^iPj "i s » = ^ for ^ = 1, . . . , n. We have proved that if G ri D then 
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there exists p G SU ch that 

> for j = 1, . . . , Ki and i = 1, . . . , fC, 

<£;=iPf =1 fori = l,...,/C, (3.27) 

XT=i^P? S l,»i = k for£ = l,...,n. 

Substituting ^ for py in (|3.27p . we get a positive solution of (|2.2p . Thus 
item 2 holds. 

Now suppose that item 2 holds, that is, there exists p G l ~ /t/c that 

satisfies ()3.27j) . Let us prove that G riD. Comparing the first and second 
line of (13371) with (pOHjh we get F(E{p)) G riD. Let z = B(p). Then 

= X>P?V} = -• 

3=1 

In the first equation we use the definition of E and in the second the third 
line of (|3.27p . Prom the definition of F it follows that F(x) = 0. Therefore, 
F(E(p)) = 0. Thus = F(E(p)) G riD and item 1 holds. ■ 
Let us recall some additional definitions from [7j. For M C M. n and 
a G 1", the translate of M by a is defined to be set 

M + a = {x + a\ xe M}. 

A translate of an affine set is another affine set. An affine set M is parallel 
to an affine set L if M = L + a for some a. Each non-empty affine set is 
parallel to a unique subspace L (see Theorem 1.2 in [7J). The dimension of 
a non-empty affine set is defined as the dimension of the subspace parallel 
to it. An (n — l)-dimensional affine set in W 1 is called a hyperplane. By 
(•,•) denote the inner product in W 1 : (x,y) = Y^i=i x iVi- Given (3 G R and 
a non-zero 6 G W 1 , the set 

# = {x| (x,6)=/?} 

is a hyperplane in M n ; moreover, every hyperplane may be represented in 
this way, with 6 and /3 unique up to common non-zero multiple (see Theorem 
1.3 in [7J). 
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For any non-zero b £ R n and any £ R, the sets 

{x| <x,6> </?}, {x|(x, &>>/?} 

are called closed half-spaces. The sets 

{x | (x,b) < /?}, {x | (x,6) > /?} 

are called open half-spaces. The half-spaces depend only on the hyperplane 
H = {x : (x, b) = /3}. One may speak unambiguously, therefore, of the open 
and closed hyperspaces corresponding to a given hyperplane. 

Let C\ and C2 be non-empty sets in W 1 . A hyperplane is said to separate 
C\ and C2 if C\ is contained in one of the closed half spaces associated with 
H and C2 lies in the opposite half-space. It is said that to separate C\ and 
C2 properly if C\ and C2 are not both actually contained in H itself. 

Now we are ready to prove Theorem 12.31 By Lemma 13.51 we have to 
show that ^ riD. 

Note that the one point set {0} is an affine set, riD is a relatively open 
convex set and (riD) D {0} = 0. Therefore, there exists a hyperplane H 
containing such that one of the open half-spaces associated with H contains 
riD (see Theorem 11.2 in [7]). Since £ H, we see that H = {x : (x, b) = 0} 
with some b £ M n , b 7^ 0. Substituting, if it is necessary, —b for 6, we see 
that there is a linear functional / : R n — ► R that sends point y G M n to value 
(y,b), and if y £ riD, then f(y) > 0. 

Recall that the state space of the Markov chain X e {m) is F{H n ). Since 
riD C OH, we see that there is a point z £ F(N n ) C Wl such that f(z) > 0. 
Also, £ F(N n ) and /(0) = 0. To prove that X e {m) is not positive recurrent 
let us apply Theorem 13.21 to the Markov chain X e {m) and the function /. 
To apply the theorem, we see that it is enough to check that 

E(f(X e (m + 1)) - f(X e (m)) | X e {m) = z) > for any z £ F(N n ) (3.28) 

To prove (|3.28j) it is enough to prove 

E(/(F(A e (m + l))) - f{F{X e (m))) \ X e (m) = x) >0 Vx £ N n (3.29) 
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To prove (|3.29p . we need some notation. For i = 1, . . . , K, denote by e)- 
the j-th coordinate vector in M Ki . By T denote the linear transformation that 
takes a point p = (pW , . . . , p^"' ) G M KlH hK ' c to the point x = (x\, . . . , x n ) G 
W 1 such that 

K. Ki 

xi = S £ j Y^v\ l) S ita t for £ = 1, . . . , n 

i=i j=i 

(i) 

In particular, T takes the point e) to the point x such that X£ = 5 e s < , for 
I = 1, . . . , n. 

Let us prove f)3.29|) . Take any x G N n . Recall that, for routing policy P, 
we have p = P(x) G Ai X • • • X A^;. Moreover, 

E(/(F(A e (m + 1))) - f(F(X e (m))) \ X e (m) = x) 

= E E A ^ + T ( e ? } ))) - 

i=l j=l 

=EE^i ) /(^(nef))) 

i=l 3=1 

= /(^(EE A 4 i)r ( e ?))) = /(Wp))) > o- 

i=l 3=1 

In the second and third equalities we use that / o F is a linear functional. 
Let us check the last inequality. We have 

F{E{p)) G F(£(Ai x • • • x A c )) = D. 

Since D is closed and convex, we see that cl(riD) = c\D = D (see the 
properties of operations ri and cl in the beginning of Section 13. 3|) . Note 
that f(y) > for y G riD and linear functional / is continuous, therefore, 
f(y) > for y G cl(riD) = D, 

Thus all conditions of Theorem 13.21 are satisfied and Theorem 12.31 is 
proved. ■ 
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